feat(ci): swap Python's OTLP encode+upload for the native Rust pipeline#1466
Open
jd wants to merge 1 commit into
Open
Conversation
This was referenced May 27, 2026
Member
Author
|
This pull request is part of a Mergify stack:
|
This was referenced May 27, 2026
Contributor
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🔴 👀 Review RequirementsWaiting for
This rule is failing.
🔴 🔎 ReviewsWaiting for
This rule is failing.
🟢 🤖 Continuous IntegrationWonderful, this rule succeeded.
🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 📕 PR descriptionWonderful, this rule succeeded.
|
3a3d6cc to
ee06c7a
Compare
380699c to
aaf35b0
Compare
Member
Author
Revision history
|
ee06c7a to
0c646f7
Compare
0c646f7 to
fa0eeaf
Compare
0e3e818 to
19c0db2
Compare
63399a1 to
73fc577
Compare
19c0db2 to
0963560
Compare
0963560 to
63a2a41
Compare
73fc577 to
64fccaf
Compare
64fccaf to
f6a515e
Compare
63a2a41 to
6faeb52
Compare
Base automatically changed from
devs/jd/worktree-rust-port/parse-junit-xml-reports-native-rust--7d0fb778
to
main
May 29, 2026 14:00
Contributor
|
@jd this pull request is now in conflict 😩 |
Second layer of the `mergify ci junit-process` port — replace the
`opentelemetry-exporter-otlp-proto-http` encode-and-upload path
in `mergify_cli/ci/junit_processing/upload.py` with a call into
a new hidden Rust subcommand, `mergify _internal junit-upload`,
that re-parses the JUnit XML files, builds the OTLP
`ExportTraceServiceRequest` with the quarantine set baked in,
gzips it, and POSTs it to
`/v1/repos/<owner>/<repo>/ci/traces`.
Two new modules under `junit_process`:
- `spans` builds one session span per upload, one suite span per
`<testsuite>`, and one case span per `<testcase>`, all sharing
a resource that carries the CI-env attributes the backend uses
for routing (provider, pipeline run id, branch, head SHA, …).
The test cases pin parent/child wiring, status-code mapping,
attribute propagation, and resource scraping by feeding a
deterministic RNG into a hidden seam. `UploadMetadata` carries
an optional `run_id` (the bridge passes the same 16-char hex
Python already printed to its UI; decoded to 8 bytes as the
session span id) and a `quarantined` name set (each matching
case span gets `cicd.test.quarantined = true`).
- `upload` POSTs the request with the same headers the Python
`OTLPSpanExporter` used to send (`Bearer` auth,
`application/x-protobuf`, `Content-Encoding: gzip`) and
matches the error wording so any log scrapers tracking Python
output keep working. Wiremock covers the happy path, the
empty-request short-circuit, and the 401 error surface.
The detector grows the resource-attribute lookups Python emits
(pipeline name, job name, run id/attempt/url, head/base ref,
head SHA, repository URL, runner name). On GitHub Actions
`pull_request` builds, `GITHUB_SHA` points at the synthetic merge
commit GitHub creates by pre-merging the PR head into the base —
not the actual code the tests ran against. The contributor's real
head SHA lives at `pull_request.head.sha` inside `GITHUB_EVENT_PATH`
and that is the value the dashboards correlate with, so
`get_head_sha` prefers the event payload and falls back to
`GITHUB_SHA` when the payload is missing, malformed, or the build
isn't a PR event. Three unit tests pin the precedence: payload
wins, missing-PR-field falls back, missing-event-file falls back.
The CircleCI PR-build API fallback Python implements stays
Python-side for now — it requires a GitHub REST API client we
don't have on the Rust side yet.
Python's `upload.upload(api_url, token, repository, spans=...)`
becomes
`upload.upload(api_url, token, repository, files=..., run_id=...,
quarantined_names=..., test_framework=..., test_language=...)`.
The caller (`process_junit_files`) extracts `quarantined_names`
from the spans the Python quarantine pass already mutated
(`cicd.test.quarantined is True`) so the Rust span builder
reproduces the same wire shape the Python pipeline used to emit.
The `opentelemetry-exporter-otlp-proto-http` runtime dep is
dropped — `opentelemetry-sdk` stays because the quarantine and
report paths still consume `ReadableSpan` attributes (those move
to native in Phase C).
`opentelemetry-proto` is the only otel dep we pull in on the
Rust side, gated to `gen-tonic-messages + trace` so it boils
down to the prost generated types plus the trace proto module —
no tonic, no otel SDK, no exporter runtime. Compression is
`flate2` and the HTTP layer reuses the existing workspace
`reqwest` to keep the TLS / rustls flavour consistent.
This is a transitional bridge. When Phase C of the port lands
later in the stack (the native `ci junit-process` orchestrator
that subsumes the entire command), the Python side of
`junit-process` and the `_internal junit-upload` subcommand
both go away. Cleanup checklist for that follow-up:
- delete `InternalSubcommand::JunitUpload`,
`InternalJunitUploadArgs`, `InternalJunitUploadOpts`,
`NativeCommand::InternalJunitUpload`, and the
`("_internal", "junit-upload")` entry in `NATIVE_COMMANDS`
- delete `mergify_cli/ci/junit_processing/upload.py`
- drop the remaining `opentelemetry-sdk` Python dep once the
rest of `process_junit_files` is gone
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Change-Id: Icbc727166711d76678877aa172ca47c2dcc07ebc
6faeb52 to
624158f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second layer of the
mergify ci junit-processport — replace theopentelemetry-exporter-otlp-proto-httpencode-and-upload pathin
mergify_cli/ci/junit_processing/upload.pywith a call intoa new hidden Rust subcommand,
mergify _internal junit-upload,that re-parses the JUnit XML files, builds the OTLP
ExportTraceServiceRequestwith the quarantine set baked in,gzips it, and POSTs it to
/v1/repos/<owner>/<repo>/ci/traces.Two new modules under
junit_process:spansbuilds one session span per upload, one suite span per<testsuite>, and one case span per<testcase>, all sharinga resource that carries the CI-env attributes the backend uses
for routing (provider, pipeline run id, branch, head SHA, …).
The test cases pin parent/child wiring, status-code mapping,
attribute propagation, and resource scraping by feeding a
deterministic RNG into a hidden seam.
UploadMetadatacarriesan optional
run_id(the bridge passes the same 16-char hexPython already printed to its UI; decoded to 8 bytes as the
session span id) and a
quarantinedname set (each matchingcase span gets
cicd.test.quarantined = true).uploadPOSTs the request with the same headers the PythonOTLPSpanExporterused to send (Bearerauth,application/x-protobuf,Content-Encoding: gzip) andmatches the error wording so any log scrapers tracking Python
output keep working. Wiremock covers the happy path, the
empty-request short-circuit, and the 401 error surface.
The detector grows the resource-attribute lookups Python emits
(pipeline name, job name, run id/attempt/url, head/base ref,
head SHA, repository URL, runner name). On GitHub Actions
pull_requestbuilds,GITHUB_SHApoints at the synthetic mergecommit GitHub creates by pre-merging the PR head into the base —
not the actual code the tests ran against. The contributor's real
head SHA lives at
pull_request.head.shainsideGITHUB_EVENT_PATHand that is the value the dashboards correlate with, so
get_head_shaprefers the event payload and falls back toGITHUB_SHAwhen the payload is missing, malformed, or the buildisn't a PR event. Three unit tests pin the precedence: payload
wins, missing-PR-field falls back, missing-event-file falls back.
The CircleCI PR-build API fallback Python implements stays
Python-side for now — it requires a GitHub REST API client we
don't have on the Rust side yet.
Python's
upload.upload(api_url, token, repository, spans=...)becomes
upload.upload(api_url, token, repository, files=..., run_id=..., quarantined_names=..., test_framework=..., test_language=...).The caller (
process_junit_files) extractsquarantined_namesfrom the spans the Python quarantine pass already mutated
(
cicd.test.quarantined is True) so the Rust span builderreproduces the same wire shape the Python pipeline used to emit.
The
opentelemetry-exporter-otlp-proto-httpruntime dep isdropped —
opentelemetry-sdkstays because the quarantine andreport paths still consume
ReadableSpanattributes (those moveto native in Phase C).
opentelemetry-protois the only otel dep we pull in on theRust side, gated to
gen-tonic-messages + traceso it boilsdown to the prost generated types plus the trace proto module —
no tonic, no otel SDK, no exporter runtime. Compression is
flate2and the HTTP layer reuses the existing workspacereqwestto keep the TLS / rustls flavour consistent.This is a transitional bridge. When Phase C of the port lands
later in the stack (the native
ci junit-processorchestratorthat subsumes the entire command), the Python side of
junit-processand the_internal junit-uploadsubcommandboth go away. Cleanup checklist for that follow-up:
InternalSubcommand::JunitUpload,InternalJunitUploadArgs,InternalJunitUploadOpts,NativeCommand::InternalJunitUpload, and the("_internal", "junit-upload")entry inNATIVE_COMMANDSmergify_cli/ci/junit_processing/upload.pyopentelemetry-sdkPython dep once therest of
process_junit_filesis goneCo-Authored-By: Claude Opus 4.7 noreply@anthropic.com